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(54) Method and apparatus for correcting degradation of book center image by image processing 
based on contour of book center image 



(57) An image processing apparatus corrects deg- 
radation of an image of a pair of spread pages of a 
bound book placed on a document table. The image 
processing apparatus includes a height obtaining unit 
which obtains a plurality of heights, above the document 
table, of a plurality of points of the pair of spread pages, 
based on a contour of the pair of spread pages which 
appears in the image, and an image correcting unit 
which corrects the degradation of the image based on 
the plurality of heights obtained by tiie height obtaining 
unit. The degradation may be unevenness of lightness, 
deformation, a blur, or the like. 
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spread pages of a bourd book as an original document, based on a contour of the pair of spread pages wfiich appears 
in an image of the pair of spread pages, where the bound book is placed on the document table; and (b) correcting deg- 
radation of the image of the pair of spread pages, based on the plurality of heights obtaned by the height obtaining unit. 
[0010] According to the second aspect of the present invention, there is provided an image processing apparatus 
5 which contains a height obtaining unit which obtains a plurality of heights, above a document table, of a plurality of 
points of a pair of spread pages of a bound book as an original document, based on a contour of the pair of spread 
pages which appears in an image of the pair of spread pages, where the bound book is placed on the document table; 
and an image correcting unit which corrects degradation of the image of the pair of spread pages, based on the plurality 
of heights obtained by the height obtaining unit. 
10 [001 1 ] With the image processing apparatus according to the second aspect of the present invention, the heights 
of a plurality of points of the book cerrter are obtained based on the contour of the book center in the image obtained 
by scanning the book cerrter. and degradation of the image is corrected based on the heights obtained from the contour 
of the book center in the image. That is, the degradation of the image can be corrected teased on only the image. There- 
fore. It is unnecessary to provide any specific sensor or additional device, or to change the construction of the conven- 
es tional scanner. Thus, the degradation of the image can be corrected at low cost 

[0012] In the image processing apparatus according to the second aspect of the present invention, the above 
height obtaining unit may contain a reference line extracting unit which extracts at least one reference line leased on a 
top and bottom portions of the contour of the pair of spread pages which appears in the image; a correction area deter- 
mining unit which determines as a correction area an area of the image corresponding to a portion of tiie pair of spread 
20 pages which is lifted from the document table, based on the at least one reference line and tiie contour; a distance 
obtaining unit which okstains a distance from each pixel of a portion of tiie contour in at least one edge of the correction 
area, to the at least one reference line; an edge height obtaining unit which obtains a height of each point of a top and 
bottom edges of the portion of the pair of spread pages above tiie document table, based on the distance obtained by 
the distance obtaining unit; and a height distribution obtaining unit which obtains a height of each point of the portion of 
25 the pair of spread pages above the document table, based on tiie height of each point of the top and bottom edges of 
the portion of the pair of spread pages. 

[0013] That is, the height of each point of the edges of the book center is obtained based on the top and bottom 
portions of the contour of the book center which appears in the image, and the height of each point of the book center 
can be obtained based on the height of each point of the edges of the book center. 
30 [0014] In the above height obtaining unit, the at least one reference line is a straight line obtained by extending an 
undeformed straight portion of the contour of tiie pair of spread pages which appears in the image. 
[001 5] The height of each point of the edges of tiie book center can be obtained even if the spread pages in the pair 
are differentiy skewed. 

[001 6] When the skew angles of the spread pages in tiie pair are identical, it is possible to obtain the distance from 
35 each pixel of a portion of the contour in at least one edge of the correction area to only one reference line. However, 
when obtaining the distance based on only one reference line, it is necessary to provide a means to equalize the skew 
angles of the spread pages in the pair. The skew angles of the spread pages on the document table are generally dif- 
ferent, and the skew angles are often different even between tiie top and bottom edges of each of tiie spread pages. 
Therefore, when the above means for equalizing the skew angles of the spread pages is not provided, it is preferable to 
40 Obtain reference lines (four in total) from the top and bottom edges of the respective spread pages in the pair, to accu- 
rately obtain the heights of the respective points of the book center above the document table. 

[0017] The correction area may be defined as an area encircled by the top and bottom portions of tine contour of 
the spread pages and lines connecting top and bottom branch points of the respective spread pages, where tiie top and 
bottom branch points are points at which the corresponding reference lines branch from the top and bottom portions of 

45 the contour, respectively. In other words, the branch points are points at which the correction is commenced. 

[0018] The height of each point of the edges of the book center can be obtained even if the book is placed on the 
document table in a slanting direction. However, when tiie image of the bound book is read by a line sensor, tiie book 
must be placed on the document table so tiiat the directions of the top and bottom edges of the book are not parallel to 
the direction of the line sensor. Otherwise, when the image of the bound book is read by an area sensor, there is no 

so requirement regarding the direction of ttie kx)ok on the document table. 

[0019] In the image processing apparatus according to the second aspect of the present invention, the above 
image correcting unit may be a unit which corrects unevenness of lightness, deformation, or a blur of the image. 
[0020] When the image correcting unit in the second aspect of the present invention corrects unevenness of light- 
ness, the image correcting unit calculates, based on tiie heights of the respective points of the book center, corrected 

55 values of lightriess for the respective pixels of the book cerrter portion of the image, where the corrected values of light- 
ness are values of lightness which tiie respective pixels of the book cerrter portion of the image will have if the book 
center of the spread pages is in contact with the document table. Then, the image connecting unit changes the values 
of lightness which the respective pixels of the book center portion of the image, to tiie corrected values of lightness. 
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table, and corrects deformation, unevenness of lightness, and a blur of an image of the bound book, based on the dis- 
tance. 

[0027] Fig. 3 is a block diagram illustrating the construction of the image processing apparatus as the embodiment 
of the present invention. As illustrated in Rg. 3. the image processing apparatus 100 as the embodiment of the present 

f invention contains an analyzing unit 106 including a reference line extracting unit 101. a correction area determining 
unit 102. a distance calculation unit 103, an edge height calculation unit 104 and a height distribution calculation unit 
105. A lightness correction unit 107. a deformation correction unit 108 and a blur correction unit 109 are connected to 
the height distribution calculation unit 105. The lightness correction unit 107, the deformation con-ection unit 108 and 
the Wur correction unit 109 are connected to a correction selecting unit 1 10. which is connected with a trimming unit 

10 111 that is connected to a skew correction unit 1 1 2. 

[0028] The analyzing unit 106 con-esponds to the aforementioned height obtaining unit in the image processing 
apparatus according to the second a^ect of the present invention. The analyzing unit 106 contains the reference line 
extracting unit 101 , the correction area determining unit 102. the distance calculation unit 103, the edge height calcula- 
tion unit 104. and the height distribution calculation unit 105, which correspond to the reference line extracting unit, the 

15 correction area determining unit, the distance detaining unit, the edge height otrtaining unit, and the height distribution 
obtaining unit, respectively That is. the analyzing unit 106 obtains a plurality of heights. above*the document table, of a 
plurality of points of the pair of spread pages of the bound booK based on a contour of the pair of spread pages which 
appears in die image of the kK>ok 

[0029] Based on the height (distance) of each point of the book cerrter portion above the document table, the light- 
20 ness correction unit 1 07 corrects unevenness of lightness in the image of the book, the deformation correction unit 108 
corrects deformation of the image, and the blur correction unit 1 09 corrects a blur in the image. The correction selecting 
unit 1 10 performs necessary correction of degradation of the image by selectively using the lightness con-ection unit 
107. the deformation correction unit 108. and the blur correction unit 109. The trimming unit 1 1 1 trims off unnecessary 
portions of the image. The skew correction unit 112 corrects the skew of the image. 
25 [0030] Explanations of the respective elements in the construction of Fig. 3 are provided below. 

[1] First, explanations on the reference line extracting unit 101 are provided below. 

Fig. 1 shows a scene of scanning of a bound book by a flat bed scanner. In Rg. 1 . the bound book 3 is placed 
on a document table 2 of the flat bed scanner as an image reading apparatus 4. In Rg. 1 . the main scanning direc- 

30 tion. the sub-scanning direction, and the direction of the lift from the document table 2 are indicated by x. y. and z 
coordinates, respectively. Considering the aforementioned case in which the skew angles of the top and tx)ttom 
portions of the contour of the pair of spread pages are different, four reference lines 11, 12. 13. and 14 (as illus- 
trated in Fig. 4) are extracted by the reference line extracting unit 101 In this embodiment. 

The image reading apparatus 4 contains a reduction optical system, and obtains density of each pixel of tiie 

35 image of the pair of spread pages in the form of digital value of n bits (e.g. 8 bits). In this embodiment, tiie digital 
value 255 corresponds to the pure white level, and tiie digital value 0 corresponds to the pure black level. Since, as 
illustrated in Fig. 2. tine density of each point of the book center portion increases with the lift of the point, the digital 
value representing the density of each point of the book center portion decreases with the lift of the point. 

When an average density value in areas of the above image corresponding to portions of the spread pages 

40 which are in contact with the document table 2 is obtained, and the entire image is binarized based on the average 
density value, the trinarized image as illustrated in Fig. 4 is obtained. In the binarized image of Fig. 4. the areas 91 
corresponding to the portions of the spread pages which are in contact with the document table 2 are white, and all 
of the other areas are black. Next, almost straight portions of the top and bottom sides of the contours tiie areas 91 
are respectively extended to obtain four straight lines as the aforementioned four reference lines 1 1 to 14. In order 

45 to determine the straight lines, any known technique for detecting a straight line, such as the Hough transformation, 

boundary tracing, and the method of least squares, may be used. 
[2] Explanations on the correction area determining unit 102 are provided below. 

Fig. 5 is a diagram illustrating an example of a contour image of a pair of spread pages of a bound book placed 
on a document table. Since a book center portion of a scanned image such as the image of Rg. 2 is darker than 

50 the aforementioned portions of the spread pages which are in contact with the document table 2, it is impossible to 

extract contours of the book cerrter portion of the spread pages from the binarized image such as the image of Fig. 
4. Therefore, the contour image of Rg. 5 is obtained by averaging the book center portion of a scanned image such 
as the image of Rg. 2. and ttien binarizing the averaged image. Thus, the contours of the book center portion of 
the spread pages can be extracted as illustrated in Fig. 5. 

55 In Fig. 5. tiie points 21. 22. 23. and 24, at which the reference lines 11. 12. 13. and 1 4 branch from the top and 

bottom portions of the contour of the spread pages, are hereinafter called correction start points. Then, an area 
encircled by a straight line connecting the correction start poirrts 21 and 23, a straight line connecting the correction 
start points 22 and 24, a portion of the contour of the spread pages between tfie correction start points 21 and 22. 
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coordinates of the points p^n and pin- 

Furthermore, the accurate value of H^^ can be obtained by the following equation (2)' 



5 Hin'= Han +{Hbn-Han)x "^^ (2)' 

where yj is the difference between the y coordinates of the points pan and Pbn. and yj is the difference between 
10 the y coordinates of the points Pan and Pin. 

Thus, the analyzing unit 106 obtains a plurality of heights, above the document table, of a plurality of points of 
the pair of spread pages of the bound book, based on the contour of the pair of spread pages which appears in the 
image of the book. 

[6] Explanations on the lightness correction unit 1 07 are provided below. 

75 Fig. 9 is a side view illustrating a geometrical relationship among the spread pages of the book document 3. 

the document table 2. light sources 5a and 5b, a mirror 8. and a charge coupled device (CCD) 9 in the flat bed scan- 
ner. In the configuration of Fig. 9. the surfaces of the spread pages of the book document 3 are illuminated by light 
from the light sources 5a and 5b, and the light reflected by the point B on the surfaces of the spread pages of tine 
book document 3 is lead through the point A on the document table 2 and is reflected by the mirror 8 to the charge 

20 coupled device (CCD) 9. In order to obtain a value of conrected lightness at the point A. intensities of the incident 
light and the reflection light at the points A and B are necessary, and can be obtained as follows. 

The intensities of tiie incident light at the points A and B are inversely proportional to the iderrtical distance from 
the light sources 5a and 5b to the points A and B. respectively 

That is, tiie intensity Ia of the incident light at the points A is expressed as 



25 



30 



I A oe1/7('V„ + /2)^ + r2 (3) 
and tine intensity Ib of the incident light at the points B Is expressed as 



Ib oc1/7(''/n + ^2+H)' + r2 (4) 



where f 1 is the thickness of the document table 2. f2 Is the distance from the centers of the light sources 5a and 5b to 
the document table 2. n is tiie refraction index of the glass of the document table 2. r is the distance from the centers of 
the light sources 5a and 5b to the midpoint p between the centers of the light sources 5a and 5b. and H is the distance 
35 from the point B to the document table 2. 

The intensities of the reflection light at tiie points A and B can be obtained by the following equations (5) and 
(6), respectively. 

That is. tiie intensity Ra of the reflection light at the points A is expressed as 
40 Ra=I A^P^2cose. (5) 

and the intensity Rb of the reflection light at the points B is expressed as 

Rg=lQxpx(C0Sv1+C0Si|»2). (6) 

45 

where p is the reflectance of ttie paper of tine book document 3, 0 is the angle at the point A between the incident light 
from the light source 5a and the reflection light, m/1 is the angle at the point B between tiie Incident light from the light 
source 5a and the normal to the surface, and \\t2 is tine angle at the point B between the Incident light from the light 
source 5b and the normal to the surface. 
so As Illustrated in Fig. 9. cose is determined according to the positions of the light sources 5a and 5b. and can 

be obtained by the following equation (7). 



cose =(f 1 +f2)/A/{r^ + (f1 +f2)^} (7) 
55 In addition, as illustrated in Fig. 9. m'I arid m;2 are expressed as 

vl=a+<I>, (S) 
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D^=DbX 2 xcosG/ (cos\t)l+costl;2 ) 




(17) 
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Therefore, the lightness at the point A can be obtained by substituting the aforementioned height Hn or Hjn* for 
H in the equation (17). Thus, the lightness at the point 6 can be corrected to the lightness at the point A. 
IS [7] Explanations on the blur conection unit 109 are provided below. 

The blur in the book center image is caused by the lift of the book center portion of the spread pages from the 
document table 2. That is, due to the lift of the book center portion, the book center portion is located beyond the 
depth of field of the reduction optical system, and therefore a blur occurs. The degrees of blurs at the respective 
points of the book center image are different according to the lifts of the corresponding points of the book center 
20 portion of the spread pages. Therefore, the blur correction unit 109 corrects the blurs of the book center image by 
varying the amount of correction according to the degrees of the blurs. 

The degree of a blur is proportional to the height of each point of the book center portion. Therefore, image 
correction processing is performed by using a Laplacean operator so that the amount of correction varies with the 
of high frequency enhancement processing using a digital Laplacean operator. In the equation (18), f(ij) indicates 
25 an image before correction, and g(i.j) indicates an Image after correction. 

g(i.j)=f(i.i)-v^f(i.j) (18) 

However, in this embodiment, a function of high frequency enhancement processing as expressed in the fol- 
30 lowing equation (19) is used, where the digital Laplacean operator is weighted. 

g(i.i)=f(i.i)-r(H)xv2f(i.j-) (i9) 

The weight y{H) in the equation (19) is a function which Is proportional to the height H. Therefore, when the height H 

35 increases, the weight y also increases. 

Thus, when substituting the aforementioned height Hp or Hjp' for H in the equation (19). correction of tiie blur 
can be performed so that the amount of tiie correction varies with the height H. 
[8] Explanations on the deformation correction unit 108 are provided below. 

The image correcting unit 108 obtains, by calculation based on the heights of tiie pluralrty of points of the book 

40 center portion, corrected pixel positions for the respective pixels of the book center portion of the image obtained 
by scanning the book document 3 placed on the document table 2. where the corrected pixel positions are imagi- 
nary pixel positions at which the respective points of the book cerrter of the spread pages will be located if the book 
center portion of tine spread pages Is In contact with the document table 2. Then, the image correcting unit 108 
moves the respective pixels of the book center portion of the Image to the corrected pixel positions, and interpo- 

45 lated values are assigned to pixels which are vacated by tine movement to the corrected pixel positions. Fig. 1 1 

shows an example of a lift of the tX)Ok center portion of the spread pages of the book document 3, and the move- 
ments of the respective points of the book center portion to the corrected positions on the document table 2. 

First, explanations on correction of deformation in edges of the correction area corresponding to the edges of 
the book center portion are provided below. 

so It is assumed that, in the correction of deformation, a plurality of points r^, (n=1 . 2. ... ) on one of the edges of 

the book center move to a plurality of conected points (n=1 . 2. ... ) on one of the reference lines. The heights 
of the plurality of points r^ (n=1. 2. ... ) can be obtained by the equation (1) based on the respective distances Lp 
(n=l . 2. ... ) from the reference line to a plurality of points of the contour corresponding to the plurality of points rn 
(n=1.2. ...). 

£5 The corrected point pn on the reference line is determined as follows. Fig. 12 is provided for explaining a way 

of determining the conected point q^ on the reference line. In Fig. 12, the y' axis corresponds to the reference line. 

As illustrated in Fig. 12. the / coordinate of the corrected poirrt q^ is determined by obtaining the correction 
amount k in the y' direction, i.e.. in the direction of the reference line. In order to obtain the correction amount K first. 
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First, an unvacated pixel near the vacated pixel p^ is searched for in the direction of the reference line 1 1 . 
When the unvacated pixel is searched for in the direction in which the y' coordinate increases, the unvacated pixel 
p2 is found. On the other hand; when the unvacated pixel is searched for in the direction in which the y* coordinate 
decreases, the unvacated pixel pi is found. When it is assumed that the densities of the unvacated pixels p^ and 
f p2 are respectively dl and d2. the distance from the pixel Pm to the pixel p^ is v1, and the distance from the pixel 

Pn, to the pixel P2 is w1 , the density value to be assigned to the pixel p^ is determined by the following equation 
(24). 

dm=d1xw1/(v1+w1)4d2xv1/(v1+w1) (24) 

10 

When the deformation of the image is corrected as above, the image of the book center portion is extended. 
Therefore, in order to preverrt overlap of the extended images of the spread pages, the extended images of the 
spread pages are relatively moved in the sub-scanning (y) direction. The amount of the relative movement of the 
extended Images of the spread pages in the y direction is obtained based on the maximum amount of the move- 
t* merrt of the pixels in the operation of correcting the deformation, i.e.. the amount of movement of the book cerrter 
potfTts 31 and 32. 

[9) Explanations on the trimming unit 1 1 1 are provided below. 

l>ie trimming unit 1 1 1 detects and trims off the fore edge portions of the image of the book document 3. Fig, 
IS shows tfie fore edge portions of the image of the book document 3. The trimming unit 111 trims off the fore edge 
.X por%on& by extracting only an image of an innermost pair of pages (corresponding to the spread pages) in the 
vna^Q of me book document 3. as explained below. 

Frst. the image of the fore edge portions is binarized by using the high frequency enhancement processing. 
Then, the t)inarized image is divided into a plurality of areas, and it is determined whether or not a straight line 
«iists 1 Mch of the plurality of areas. Since edges of the respective pages of the book document appear as a 
sira*g^ hne in some areas which overlap the fore edge portions, the trimming unit 111 recognizes that the image 
of the desired pair of spread pages extends inside the innermost areas in which a straight line appears, and 
extracts only a region inside the innermost areas as the image of the desired pair of spread pages. In order to 
detect a straight line, the Hough transformation, boundary tracing, the method of least squares, or the like may be 
used 

20 [101 Other matters 

The lxx>k document 3 may be placed on the document table 2 in a slanting direction. Therefore, the image of 
the t)OOk document 3 may be skewed. Further, skew angles of the right and left pages may be different. In such 
cases, the appearance of an output image is deteriorated. Therefore, skew correction processing is performed by 
the skew correction unit 112. Even if the skew angles of the right and left pages are different, the skew correction 
25 unit 1 12 can obtain the skew angles based on the reference lines 11 to 14. The skew of the image of each page 
can be independently corrected by rotation processing such as the Affine transformation. 

[0031 ] As described above, in the above embodiment, correction of degradation of an image can be performed by 
only image processing, where the image is an image of a pair of spread pages of a book document 3 placed on a doc- 
40 ument table 2, and obtained by an image reading device having a reduction optical system. The degradation of the 
image may include unevenness of lightness, deformation, a blur, or the like, and occurs in a book center image. The 
correction is performed by only image processing, based on heights or Hj^' of a plurality of points of the book center 
image. 

[0032] Although, in the described emlaodiment, the operation of detecting the heights of the respective points of the 
45 book center portion is incorporated in the operation of correcting the degradation of the image, the operation of obtain- 
ing the heights per se can be used separately. 

[0033] All of the contents of the Japanese Patent Application No. 1 1(1999)-44965 are incorporated into this speci- 
fication by reference. 

so Claims 

1 . An image processing apparatus comprising: 

a height obtaining unit which obtains a plurality of heights, above a document table, of a plurality of points of a 
55 pair of spread pages of a bound txjok as an original document, based on a contour of the pair of spread pages 

which appears in a scanned image of the pair of spread pages, where the bound book is placed on the docu- 
ment tat}le: and 

an image correcting unit which corrects degradation of the image of the pair of spread pages, based on the plu- 
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